Papers with semi-automated approach
Efficient, Uncertainty-based Moderation of Neural Networks Text Classifiers (2022.findings-acl)
Copied to clipboard
| Challenge: | A series of benchmarking experiments based on three different datasets and three state-of-the-art classifiers show that our framework can improve the classification F1-scores by 5.1 to 11.2% (up to approx. 98 to 99%) |
| Approach: | They propose a semi-automated approach that passes unconfident, probably incorrect classifications to human moderators to minimize the workload. |
| Outcome: | The proposed approach can improve the classification F1-scores by 5.1 to 11.2% (up to approx. 98 to 99%) while reducing the moderation load up to 73.3% compared to a random moderation. |
“Judge me by my size (noun), do you?” YodaLib: A Demographic-Aware Humor Generation Framework (2020.coling-main)
Copied to clipboard
| Challenge: | Humor is subjective and can be interpreted in different ways by different people. |
| Approach: | They propose an automatic method for filling the blanks in Mad Libs stories . they build upon the BERT platform to predict location-biased word fillings in incomplete sentences . |
| Outcome: | The proposed framework outperforms a semi-automated approach for filling the blanks in Mad Libs stories while accounting for the demographic backgrounds of the desired audience. |
SoFA: Shielded On-the-fly Alignment via Priority Rule Following (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing alignment methods fail to adapt to the diversity of preferences and regulatory standards. |
| Approach: | They propose a method for prioritizing rules over user instructions to minimize misalignments in Large Language Models. |
| Outcome: | The proposed approach minimizes misalignments and adapts smoothly to various unseen rules, ensuring they are shielded from hijacking and that the model responds appropriately. |
Developing PUGG for Polish: A Modern Approach to KBQA, MRC, and IR Dataset Construction (2024.findings-acl)
Copied to clipboard
Albert Sawczyn, Katsiaryna Viarenich, Konrad Wojtasik, Aleksandra Domogała, Marcin Oleksy, Maciej Piasecki, Tomasz Kajdanowicz
| Challenge: | Existing KBQA datasets are outdated and inefficient in human labor, and assisting tools like Large Language Models (LLM) are not utilized to reduce the workload. |
| Approach: | They propose a semi-automated question answering task that uses structured knowledge graphs to answer extensive knowledge-intensive questions. |
| Outcome: | The proposed approach includes KBQA, MRC, and Information Retrieval tasks for low-resource languages. |
Do Current Video LLMs Have Strong OCR Abilities? A Preliminary Study (2025.coling-main)
Copied to clipboard
| Challenge: | a new benchmark evaluates video-based optical character recognition (Video OCR) performance of multi-modal models in videos . the benchmark aims to improve video LLMs' ability to extract text from video content . previous benchmarks have focused on video QA, but not video-related QA. |
| Approach: | They propose to evaluate the video OCR performance of multi-modal models in videos . they use a semi-automated approach that integrates the OCR ability of image LLMs with manual refinement . |
| Outcome: | The proposed benchmark includes 1,028 videos and 2,961 question-answer pairs . it integrates the OCR ability of image LLMs with manual refinement . |
Can LLMs Help Uncover Insights about LLMs? A Large-Scale, Evolving Literature Analysis of Frontier LLMs (2025.acl-long)
Copied to clipboard
| Challenge: | Recent surveys of literature highlight the overwhelming growth of Large Language Models (LLMs). |
| Approach: | They propose a semi-automated literature analysis approach that automates literature analysis using LLMs. |
| Outcome: | The proposed approach reduces paper surveying and data extraction by 93% compared to manual methods. |
Exploring the Numerical Reasoning Capabilities of Language Models: A Comprehensive Analysis on Tabular Data (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent benchmarks have assessed language models' numerical abilities . limitations include tokenization and representation of numbers in text, hallucination, and a lack of numerical commonsense knowledge. |
| Approach: | They propose a hierarchical taxonomy for numerical reasoning skills that includes representation, number sense, manipulation, and complex reasoning. |
| Outcome: | The proposed model outperforms other models on the tabular Natural Language Inference task. |
HATECAT-TR: A Hate Speech Span Detection and Categorization Dataset for Turkish (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a new dataset of Turkish tweets contains 4465 hateful spans . each hateful post is directed at one of eight minority groups . |
| Approach: | They propose a span-annotated dataset of Turkish tweets containing 4465 hateful spans . each hateful spat is categorized into one of five discourse types . |
| Outcome: | The proposed dataset contains 4465 hateful spans across 2981 tweets . each span is categorized into one of five discourse types . |